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METHOD AND APPARATUS OF MUTLIPLEXING MEDIA STREAMS 



RELATED APPLICATIONS 

[001] This application claims the priority of U.S provisional application no. 60/421,247, entitled 
"Statistical Muxing of MPEG streams using partial encryption", filed 25 October 2002. 

BACKGROUND 
Compression of media streams 

[002] Methods and apparatuses for compressing and transmitting media signals are known in 

the art. Compressed digital video is largely becoming the preferred medium to transmit to 
video viewers everywhere. 

[003] Part of the Moving Pictures Experts Group (MPEG) specifications are standardized 

methods for compressing and transmitting video. Various audio compression techniques 
are also known in the art. In general, MPEG is used today for transmitting video over 
terrestrial, wireless, satellite and cable communication channels and also for storing 
digital video. MPEG includes many compression schemes including MPEG-2, MPEG-4, 
MPEG-7. 

[004 ] An audio stream is organized as an ordered sequence of frames. A video stream is usually 
organized as an ordered sequence of frames, each frame includes a plurality of pixels. A 
video frame includes a plurality of slices, each slice including a plurality of macro blocks. 
The audio and video streams are provided to an audio encoder and video encoder 
respectively to generate compressed audio and video elementary streams, also referred to 
as elementary streams. 

[005] MPEG compression/encoding utilizes various compression schemes, such as adaptive 
quantization, intra-frame encoding, inter-frame encoding, run length encoding and 
variable length coding. Intra-frame coding takes advantage of spatial redundancies in a 
picture. Inter-frame coding takes advantage of temporal redundancies from picture to 
picture in a video sequence. Inter- frame coding involves motion estimation and motion 
compensation. There are three types of motion estimations - forward, backward and bi- 
directional. MPEG-2 defines macroblocks are the elementary unit for motion 
compensation and adaptive quantization. Each macroblock is associated with a 
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quantization factor field (Q scale), representative of the degree of quantization. A slice, 
including a plurality of macroblocks includes a slice header that has a quantization factor 
field that is associated to some of the macro blocks of the slice. 

[006] According to MPEG-2 the compressed elementary streams usually include a sequence of 
three types of frames. These types are known as I-frame, P-frame and B-frame. I-frames 
use only intra-coding. P-frames use forward prediction and usually also intra-coding. B- 
frames use bi-directional coding (forward and/or backward prediction) and optionally also 
intra-coding. In a sequence of I, P, and B-frames, each P-frame is encoded in view of a 
previous I-frame or P-frame. Each B-frame is coded using a previous I-frame of P-frame 
and/or a next I-frame or P-frame. 

[007] An MPEG-2 recognizable frame can be reconstructed from an I-frame alone, but not from 
a B-frame alone. Only I-frames and P-frames can be anchor frames that are used to 
predict other frames. I-frames allow for reconstructing a recognizable frame but offers 
only relatively moderate compression. B-frames are usually much smaller than I-frames. 
Each frame includes a frame header that includes a frame type indication, indicating 
whether the frame is an I,B or P frame. 

[008] MPEG-2 Frames are sometimes arranged in groups that are referred to as Group Of 

Pictures (GOP). Usually, each GOP starts by an I-frame that is followed by B-frames and 
P-frames. 

Transmission of compressed media streams 

[009] After a video stream or an audio stream is compressed it is further packetized (and 
optionally multiplexed with other streams) to facilitate transmission over networks. 
MPEG-2 standard describes a transmission apparatus that receives video and audio 
elementary streams, packetizes these elementary streams to provide PES packets and then 
converts the PES packets to Transport Stream packets or Program Stream packets. 
Transport stream packets may be transmitted over a network or stored in a storage unit. 

[ 0 0 1 0 ] A Transport Stream combines one or more programs with one or more independent time 
bases into a single stream. Transport Streams include transport stream packets of 188 
bytes. Transport Stream packets start with a transport stream packet header. The header 
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includes a packet ID (PID). Transport Stream packets of one PID value carry data of a 
single elementary stream. 
[0011] Transport Streams may be of either fixed or variable bit rate. Some programs of the 
Transport Stream are of a variable bit rate. A typical variable bit rate program can be 
generated by allocating more bits to complex scenes, and allocating less bits to more 
simple scenes. 

[0012] Transport Streams are provided to a channel of a limited available bandwidth/storage 
space. The ISO/IEC 13818-1 specification defines a channel as a digital medium that 
stores or transports a Transport or a Program Stream. The aggregate bandwidth of all the 
components of the Transport Stream must not exceed, at any time, the available 
bandwidth allocated to the transmission of the transport stream over the channel. 

[0013 ] Various lossy and lossless techniques are implemented to perform rate shaping. These 

techniques are also known as statistical multiplexing rate shaping. U.S. patents 6,038,256 
and 6,192,083 of Linzer et al, U.S. patents 5,862,140 and 5,956,088 of Shen et al and U.S 
patent 5,877,812 of Krause et al, describe some of these prior art methods. Lossless 
techniques usually do not require further compressing of media pictures. Lossless 
techniques usually involve altering the timing of transmission of transport packets. For 
example, lossless techniques may involve changing a transmission timing of media 
stream components by either delaying or advancing a transmission of transport packets. 
Lossy techniques involve additional compression, and are usually implemented whenever 
the appliance of lossless techniques is not feasible or does not provide sufficient results. 
The additional compression usually results in visual quality degradation. An apparatus 
and method for rate shaping statistical multiplexing is also described in U.S patent serial 
number 6434141 titled " Communication management apparatus and method" of Oz et al, 
which is incorporated herein by reference. 

Conditional access 

[0014 ] In order to allow only specific viewers to watch transmitted streams, conditional access 
apparatuses and methods are utilized. Conditional access apparatuses may scramble 
information and/or encrypt them. Encryption keys are also sent to conditional access 
devices, such as those that are included within client devices, to enable decryption of the 
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media streams. As an example, according to the MPEG-2 standard, each Transport packet 
is encrypted and/or scrambled either individually or in conjunction with other transport 
packets. Two bits are defined in the MPEG-2 Transport packet header as 
transport_scrambling_control bits. If the value of these two bits is different than '0' then 
the payload of the packet is scrambled. For convenience of explanation the application 
refers to encryption while it applied mutatis mutandis to scrambling. 
[0015] U.S patent application number 2002/0196939 of Unger et al. titled "Decoding and 
decryption of partially encrypted information", U.S patent application number 
2003/0026423 titled "critical packet partial encryption", U.S patent application 
2003/0021412 of Candelore et al. titled "Partial encryption and PID mapping", and U.S 
patent application 2003/0046686 of Candelore et al., titled "time division partial 
encryption" describe apparatuses and method for allowing coexistence of multiple 
conditional access encryption apparatuses that partially encrypt a television program. The 
methods and apparatuses duplicate packets to be encrypted to provide shadow packets 
that are assigned special PID values so that when an encrypting unit receives the media 
stream it is able to recognize those packets that are to be encrypted. These patent 
applications are incorporated herein by reference. 

Multiplexing and encrypted streams 

[0016] Statistical multiplexing rate shaping involves some level of decoding applied to the 

programs that form the Transport Stream. This, generally, requires stripping the MPEG 
Transport layer, manipulating the MPEG Video layer and then re-multiplexing the 
programs in MPEG Transport packets. These operations require that the statistical 
multiplexers are able to read the media streams or at least a portion of the media stream. 
Lossless techniques that alter the timing of transmission of transport packets may handle 
transport packets that include non-encrypted timing information. Lossy techniques 
usually compress the payload of a transport packet, thus require that the payload is not 
encrypted. Thus, in many cases, encrypted media streams cannot be rate shaped 
multiplexed at their encrypted form. 

[ 0 0 1 7 ] In many cases the media streams are encrypted before they reach the statistical 

multiplexer. This happens, for example, both with Video On Demand streams and with 
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media streams arriving via satellite. Once the media streams are encrypted, statistical 
multiplexing rate-shaping requires decryption of the encrypted media streams, subject 
them to statistical multiplexing rate-shaping and then encrypt them. In many cases the 
statistical multiplexers are not provided with decryption keys, for various reasons 
including security related reasons. Thus, these statistical multiplexers are not able to 
decrypt the incoming encrypted streams. Even if the encryption keys are available to the 
statistical multiplexers the procedure of decrypting and encrypting is very complex, 
especially as advanced high security encryption algorithms are used. 
[0018] There is a need to provide a method and an apparatus that facilitates both statistical 
multiplexing and conditional access. 
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SUMMARY OF THE INVENTION 

[0019] The invention provides an electronically readable medium, an apparatus and a method for 
generating a multiplex of media streams, the apparatus and method are capable of 
receiving a set of media streams that includes first type media stream components and 
second type media stream components; applying a modification process that is not 
adapted to modify second type media stream components, such as to provide at least one 
modified first type media stream component; and providing a multiplex that includes 
second type media stream components, modified first type media stream components and 
any remaining non-modified first type media stream components. 

[0020] According to an embodiment of the invention second type media stream components are 
encrypted in a manner that does not facilitate their modification by the modification 
process. It is noted that the modification scheme can change, and that the change may 
differ the criteria applied for selecting between first and second type media stream 
components, whereas the selection is followed by sending the first type media stream 
components to be modified. 

[0021] According to an embodiment of the invention, the apparatus and method are capable of 
handling at least one media stream that includes first and second type media stream 
components. This at least one media stream is also represented by multiple 
representations (e.g., layers). The multiple layers can be generated by that apparatus or 
method, but this is not necessarily so. 

[0022] According to various embodiments of the invention the multi layers may provide 

temporal scalability or spatial scalability, they can be generated by applying multiple 
quantization levels and/or by applying filtering or manipulation. Temporal scalability is 
known in the art and may include dropping portions of a media stream. 

[0023] According to an embodiment of the invention an electronically readable medium, an 
apparatus and a method are capable of generating at least one control parameter in 
response to the modification process and use the at least one control parameter to 
determine how media streams shall be represented, modified, or encrypted. 

[0024] According to yet further embodiments of the invention there are provided electronically 
readable medium, a method and an apparatus for partially encrypting a media stream that 
in turn is represented by a multiple layers. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[ 0 02 5 ] In order to understand the invention and to see how it may be carried out in practice, a 

preferred embodiment will now be described, by way of non-limiting example only, with 

reference to the accompanying drawings, in which: 
[0026] Figure 1 illustrates an apparatus for generating a multiplex of media streams, according to 

an embodiment of the invention; 
[0027] Figures 2a-2c illustrate apparatuses for generating a multiplex of media streams, 

according to some embodiments of the invention; 
[0028] Figure 3 is a flow chart of a method for generating a multiplex of media streams, 

according to other embodiments of the invention; and 
[0029] Figures 4-5 illustrate methods for partial encryption, according to embodiments of the 

invention. 
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DETAILED DESCRIPTION 

[0030] For simplicity of explanation the following detailed description refers to second type 
media stream components. It is noted that when modification processes are applied to 
certain media stream components time shift may result in other media stream 
components. Hence, some media stream components may be treated as second type media 
stream components with regard to time shifting. For example, if timing constraints related 
to a media stream component are unknown due to scrambling or encryption (not 
necessarily of that specific component) then such a component can be treated as a second 
type media stream component with regard to time shifting modification. On the other 
hand if the size of the media stream components may be modified then media stream 
components that are scrambled or encrypted in a manner that does not allow size 
modifications (for example transport stream packets that have encrypted payloads) are 
treated as second type media stream components with regard to size modification. 

[0031] The term "media stream components" is a sequence of signals that represents media 
related information. It may include video as well as audio data and may also include 
transport packets or other types of packets or frames. 

[0032 ] Figure 1 illustrates an apparatus 10 for generating a multiplex of media streams, in 

accordance with an embodiment of the invention. For simplicity of explanation various 
components, such as storage units, voltage supply units, receivers and transmitters are not 
illustrated. 

[0033 ] Apparatus 10 includes an interface 12, statistical multiplexing unit 14, multiplexing unit 
18, an intermediate unit 16 and controller 20. The controller 20 is capable of controlling 
at least some of units 12-18 and may also participate in some processing tasks performed 
by apparatus 10. For example, controller 20 can execute an optional task of providing at 
least one control parameter in response to signals from unit 14, or other units. 

[0034] The partial encryption may include partial encryption in manners known in the art. This 
includes the partial encryption schemes described in the mentioned above U.S patent 
applications number 2002/0196939, 2003/0021412, 2003/0046686 and especially those 
described in U.S patent application 2003/0026423, all being incorporated herein by 
reference. 



i 
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[0035] These U.S patent applications describe the following partial encryption schemes:(i) 
encrypting only audio streams,(ii) encrypting programs on a time division basis, (iii) 
encrypting N packets of a sequence of M packets whereas M>N, whereas M and N can be 
fixed, or can be given random pseudo-random or semi-random values; (iv) encrypting 
certain +video frames of a program; and (v) encrypting packets based upon their 
importance to a proper decoding of a program. The latter scheme can include encrypting 
only I-frames, encrypting "start of frame" transport stream packets that include PES 
headers or other headers, and the like. 

[0036] According to other embodiments of the invention partial encryption of a media stream is 
possible when the media stream is represented by multiple layers. Partial encryption may 
involve encrypting one or more representation or layer, while not encrypting one or more 
other layers or representations. 

[0037] According to various embodiments of the invention the multiple representations or layers 
may provide temporal scalability or spatial scalability, they can be generated by applying 
multiple quantization levels and/or by applying filtering or manipulation. Filtering can 
involve filtering a frequency domain transformation (such as discrete cosine transform) 
coefficients of a video stream. 

[0038] Generating multiple representations of a media stream can include: (i) quantizing the 
media stream according to multiple quantization levels (Q scales) that differ from each 
other, to provide multiple quantized layers, and (ii) converting the multiple quantized 
layers to a base layer (the most coarsely quantized layer) and to at least one supplemental 
layer. Whereas supplemental layers are defined between each pair of consecutive 
quantized layers. An apparatus and method for providing a base layer and one or more 
supplemental layers is described at patent application serial number 09/461,859, filed at 
December 15, 1999, titled "method and apparatus for transmitting media streams over a 
variable bandwidth network", which is incorporated herein by reference. According to 
various embodiments of the invention one or more layers of the base layer and/or the 
supplemental layers are encrypted or partially encrypted, while one or more other layers 
are not. 

[0039] According to yet another embodiment of the invention at least one media stream of the 
multiple media streams is processed to provide spatial resolution scalability. Thus, the 
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media stream is represented by multiple layers that may be reconstructed to provide 
media stream of varying spatial resolution. A method for providing such a representation 
is described at U.S. patent 6,057,884 of Chen that is incorporated herein by reference. 
Partial encryption of such a media stream may involve encrypting only one or more layers 
while not encrypting another layer or other layers. Partial encryption may also involve 
encrypting a portion of at least one layer. 

[0040] Interface 12 receives multiple media streams. Interface 12 may select a set of media 
streams out of said multiple media streams to be multiplexed, according to various 
parameters including viewers request, various policy rules and the like. Figure 1 
illustrates multiple (Y) partially encrypted media streams, but this is not necessarily so. In 
the figure all streams are partially encrypted. It is noted that more than a single media 
stream can be partially encrypted, and one or more media streams may be fully encrypted. 

[0041] Interface 12 processes the received media streams components that form the media 

streams and determines which media components are encrypted and which are not. This 
determination may involve analyzing the media stream components, analyzing some of 
the media stream components or analyzing information associated with the media stream 
components. Some media stream components can even be dropped for various reasons, 
such as belonging to a media stream that will not form a part of the multiplexed output 
signal of apparatus 10. 

[0042] The non-encrypted media stream components are sent to a statistical multiplexing unit 14 
that has rate shaping capabilities. 

[0043] Each media stream component may include an encryption indication, but multiple media 
stream components may be associated with a single encryption indication. For example, a 
MPEG-2 transport stream packet header includes transport_scrambling_control bits that 
indicate whether the transport stream packet includes encrypted information or not. 

[0044] The statistical multiplexing unit 14 can modify the size and/or timing (perform time 
shifting) of non-encrypted components to provide smaller modified media stream 
components and/or time shifted modified or non-modified media stream components. 

[0045] Statistical multiplexing unit 14 modifies non-encrypted media segments according to 

various parameters such as a bit rate limitation. The bit rate limitation usually reflects the 
amount of available bit rate for the transmission of the media streams. 
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[0046] For example, the modification may be responsive to a bit rate target that represent 
available bit rate for the transmission of the multiplex signal over a communication 
channel. The communication channel can be connected to end users, but this is not 
necessarily so as it can be a part of a distribution network. The multiplex may be further 
re-multiplexed and multiplexed until it reaches its destination. 

[ 0 04 7 ] In many cases the available bit rate for transmitting (or storing) the multiplex is constant, 
but this is not necessarily so. The available bit rate can fluctuate or vary over time. 

[0048] Statistical multiplexing unit 14 may modify media stream components in response to 

various parameters and control schemes which may include supporting quality of service 
rules, allocating available bit rates to media streams according to the media stream 
transmission priority level, and the like. 

[0049] Statistical multiplexing unit 14 outputs a multiplexed signal that is then re-multiplexed by 
multiplexing unit 18 with the encrypted media stream components from intermediate unit 
16 such as to provide an output multiplexed stream. 

[0050] According to another aspect of the invention, instead of performing a rate shaping 

statistical multiplexing by a first unit and then performing multiplexing by a second unit, 
both units or operations can be combined. For example, statistical multiplexing unit 14 
can receive both encrypted and non-encrypted media stream, components and perform 
statistical multiplexing on both types, while not applying rate shaping techniques on the 
encrypted media stream components. Yet according to another example, the two units 
include a modifying unit that is controlled by a multiplexing unit. 

[0051] The rate shaping process is applied in response to bit rate limitations that is responsive to 
the bit rate of the non-encrypted and encrypted media stream components. 

[0052] The encrypted media stream components can be directly provided from interface 1 2 to 
multiplexing unit 18, can pass through an optional intermediate unit 16, and can even be 
sent to the statistical multiplexing unit 14 where they are not modified. Intermediate unit 
16 may be adapted to provide the statistical multiplexing unit 14 an indication about the 
size and timing of the encrypted media stream components, but this is not necessarily so 
as the statistical multiplexing unit 14 may estimate the size or receive it from another 
source. 
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[0053 ] The modified non-encrypted media stream components as well those non-encrypted 
media stream components that were not modified and the encrypted media stream 
components are provided to multiplexing unit 18 that multiplexes these media stream 
components to provide an output multiplex. 

[0054 ] According to an embodiment of the invention apparatus 10 operates in sessions. Even if it 
constantly receives media streams is operates on a group of media stream components 
basis. The group may include media stream components that are received during a certain 
time period, media stream components that are supposed to be transmitted during a 
certain time period, a group that includes a certain amount of media stream components 
and the like. 

[ 0 0 5 5 ] To allow such a group based operation the apparatus has buffering capabilities that are 
not illustrated for convenience of explanation. 

[0056] Figures 1, 2a and 2b provide an example of multiple media stream components that are 
processed by apparatus 10. It is noted that the amount and identity of media stream 
components per media stream, the amount of media streams, the size of each media 
stream component, the amount and identity of media stream components that are 
processed per modification session, the relationship between the various media stream 
components (encrypted, modified non-encrypted and non-modified non-encrypted media 
stream components) does not limit the scope of the invention. For example, these figures 
illustrate media streams that have the same amount (Z) of media streams components, that 
the modification process does not reduce the amount of media stream components and 
that same amount of media stream components (K)are processed per session. All these 
assumptions do not necessarily reflect real situations, and are used only to simplify the 
explanation. Furthermore, these examples do not describe time shifting of media stream 
components, while such a time shift can be implemented by unit 14. 

[0057] Referring to Figure 1, multiple media stream components are provided to interface 12. 
They include Y media streams, each including Z media stream components. The first 
media stream components of a first media stream is denoted 50i,i and the last (Z th ) media 
stream component of the last (Y th ) media stream is denoted 50 Y ,z. It is further assumed 
that the media streams are partially encrypted. Encrypted media stream components are 
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illustrated by full boxes while non-encrypted media stream components are illustrated by 
empty boxes. 

[0058] Each session determines how to modify and multiplex a group of K media stream 

components from each media stream: 50i f i - 50i,k • • • 50y,i - 50y,k- These media stream 
components are associated with timing information and are supposed to be provided to a 
target decoder or processor during a certain time period. 

[0059] It is also assumed that the target bit size, TBS, that is allocated to the group is already 
known or estimated. 

[0060] While media streams components 50i,i - 50i,z ... 50y,i - 50 Y ,z are received by interface 

12 (or previously received and stored to be retrieved) only a portion of these media stream 
components are processed during a single session. Interface 12 analyzes the group of 
received media stream components to determine which are encrypted and which are not. 
The non-encrypted media stream packets of the group are sent to statistical multiplexing 
unit 14, while the encrypted media stream packets of the group are sent to intermediate 
unit 16 that determines the encrypted media stream component aggregate size ES and 
provides ES to the statistical multiplexing unit 14. 

[0061] The rate shaping is responsive to bit rate limitations. For example, statistical multiplexing 
unit 14 can deduct the encrypted media stream component aggregate bit rate, ES, from the 
target bit rate TBS to determine the aggregate bit rate, NES, that can be allocated to non- 
encrypted media stream components. If NES is smaller or equal to TBS - ES then 
additional compression is not required. Else, statistical multiplexing unit 14 must further 
perform rate shaping operations. 

[0062] The statistical multiplexing unit 1 4 may apply various statistical rate-shaping methods 
known in the art. 

[0063] The statistical multiplexing unit 14 may also be responsive to modifications priorities 
assigned to media streams. Exemplary priority-based methods statistical multiplexing 
methods are described in U.S. patent application serial number 09/870069 titled "method 
and apparatus for prioritized bit rate conversion" filed 29 May 2001, U.S. patent 
application serial number 09/870056 titled "method and apparatus for comparison-based 
prioritized bit rate conversion" filed 29 May 2001, and U.S patent application serial 
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number 10/01 1341 titled "method for generating a multiplexed sequence of media units" 
filed 12 November 2001, all being incorporated herein by reference. 

[ 0 0 64 ] By responding to modification priorities the statistical multiplexing unit 14 will modify 

high modification priority media stream components before modifying low priority media 
stream components are modified. 

[0065] The statistical multiplexing unit 14 may modify media stream components more than 
once till it reaches the target bit size. 

[0066] The statistical multiplexing unit 14 does not necessarily modify all the non-encrypted 
media stream components. It may output both modified media stream components 
(illustrated as partially filled boxes) as well as non-modified media stream components 
that were not modified during the modification process. 

[0067] According to an embodiment of the invention indications of the modification process may 
be used to provide a feedback signal. This feedback signal represents at least one control 
parameter that may affect the encryption level of the media streams. An encryption level 
may reflect the security level of the media stream but in many cases it also affects the size 
of the encrypted media stream. If, for example, additional bit rate can be allocated for 
transmission of media streams then at least some of this additional bit rate can be used to 
encrypt more media stream components. 

[ 0 0 6 8 ] In deviation from encryption of MPEG-2 transport stream packets, the additional bit rate 
can also be used for encrypting media stream components with more size consuming 
encryption methods. This additional bit rate can be allocated on a priority level basis. 

[0069] Referring to partial encryptions schemes described in U.S patent application 

2003/0026423, they can modified in the following manners: (i) if partial encryption 
involves encrypting only audio streams, then video may also be encrypted and/or the 
audio can be encrypted more extensively; (ii) if programs are encrypted on a time division 
basis, then lengthier portions of the programs may be encrypted; (iii) if N packets of a 
sequence of M packets are encrypted then N can be increased and/or M can be decreased; 
(iv) if certain MPEG video frames of a program are encrypted then more frames may be 
encrypted; and (v) if packets are encrypted based upon their importance to the video 
quality of a program then lesser important packets may be encrypted. For example, B- 
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frames or P-frames can be encrypted in addition to the previously encrypted I-frames. 
Any combination of these schemes may also be implemented. 

[0070] The allocation of additional bit rate for encryption (as encrypted media stream 

components are not modified) can be responsive to an encryption priority assigned to the 
media stream components. For example, high encryption priority media stream 
components may be encrypted, re-encrypted with another encryption scheme or even 
encrypted by more than a single encryption scheme before low encryption priority media 
stream components are encrypted. 

[ 0 0 7 1 ] It is noted that a single media stream can be encrypted in multiple manners, to allow 
decryption by various decryption apparatuses, including legacy conditional access 
apparatuses. 

[0072] Figure 2a - 2c illustrate three embodiments in which apparatus 10 of figure 1 is connected 
to a selection unit 13, to an encrypting unit 1 1 or to a media stream processing unit 15. 
These figures illustrate a single media stream, for simplicity of explanation, although the 
apparatus is configured to process many media streams concurrently. 

[0073] Figure 2a illustrates apparatus 10 that is connected to selection unit 13. This figure 
illustrates a selection unit that has to select between three versions of media stream 
components (high, medium and low level encryption level) of a first session media stream 
components 50i,k+i - 50^ in response to at least one control parameter that are 
determined after a first session media stream components 50i,i - 50i, k are processed. The 
at least one control parameter may select between the versions or may be further 
processed before the selection is made. For example, the at least one control parameter 
may indicate LOW, MEDIUM or HIGH encryption, but may also reflect an available bit 
rate, that is compared to the size of the three versions before a selection is made. 

[0074] Figure 2b illustrates apparatus 10 that is connected to an encrypting unit 1 1 . This figure 
illustrates a session based selection scheme in which media stream components 50i,i - 
50i jk are processed during a first session (denoted "A") and at least one control parameter 
for a second session is determined (denoted "B"). The figure illustrates an example where 
an increase in allocated bit rate facilitates generating second session media stream 
components 50i,k+i - 50^ (denoted "C") that is more encrypted (for example - 
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encrypted in a more secure method) then the first session media stream components 
sequence. 

[0075] Figures 2a and 2b describe a method in which the at least one control parameter is 
updated on a modification session basis, but this is not necessarily so. 

[0076] Figure 2c illustrates an apparatus 10 that is connected to a media stream processing unit 
15 and encrypting unit 11. The media stream processing unit 15 is capable of generating a 
base layer and multiple supplemental layers, and supplying at least one of the layers to the 
encrypting unit 1 1 . The encrypting unit 1 1 encrypts at least a portion of one layer while 
not encrypting a portion of one or more other layers. The encryption unit 1 1 may have its 
own interface for receiving the media streams. 

[0077] According to an embodiment of the invention the at least one control parameter is sent to 
a controlling unit (not shown) that controls the encrypting unit and/or to the media stream 
processing unit. The controlling unit may specify the PED of media stream components to 
be encrypted so that the encrypting unit is capable of recognizing which components to 
encrypt. The at least one control parameter may also be used to alter the representation of 
a media stream, such as to determine the amount of layers or the content of the layers. 

[0078] According to an embodiment of the invention the media stream processing unit 1 5 may be 
adapted to provide multiple layers representative of the media stream. According to 
various embodiments of the invention media stream processing unit 15 may generate 
multiple layers such as to provide temporal scalability or spatial scalability, it may also 
generate layers by applying multiple quantization levels and/or by applying filtering or 
manipulation. 

[0079] The media stream processing unit 15 may convert a media stream to provide multiple 
layers or even convert a first set of layers to a second set of layers. 

[0080] According to an embodiment of the invention the media stream processing unit 15 is 

capable of quantizing a media stream according to multiple quantization levels to provide 
multiple quantized layers. The most coarsely quantized representation of the media 
stream is defined as a base layer. The media stream processing unit 15 is further adapted 
to produce a supplemental layer for each successive pair of said quantized layers. 

[0081] The encrypting unit 1 1 may be adapted to encrypt at least a portion of a first layer (being 
a base layer or a supplemental layer). 
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[0082 ] According to another embodiment the media stream processing unit 15 is adapted to: (i) 
down-sample the media stream to provide a first spatial resolution layer; (ii) up-sample 
the first spatial resolution layer to provide a first reconstructed layer, and (iii) to produce a 
first enhancement layer that reflects the difference between the media stream and the first 
reconstructed layer . The encrypting unit 1 1 may be adapted to encrypt at least a portion 
of the first enhancement layer or at least a portion of the first spatial resolution layer. It is 
noted that spatial scalability is not limited to two layers, and that the media stream 
processing unit may provide more multiple spatial resolution layers and enhancement 
layers. 

[0083] The encrypting unit 1 1 is adapted to encrypt at least a portion of an enhancement layer or 
at least a portion of the first spatial resolution layer. 

[0084] According to yet a further aspect of the invention the partial encryption scheme can be 

altered (for example, by media stream processing unit 15) by changing the representation 
of a media stream in a multiple media stream representation scenario. For example, if 
additional bandwidth can be allocated to transmit a certain media stream then more layers 
of said stream may be transmitted. The transmission of more layers may result in a need 
to alter the encryption of the transmitted layers and/or may inherently change the 
encryption of the media stream as layers of differing encryptions are transmitted. 

[0085] Figure 3 is a flow chart of method 100 for generating a multiplex of media streams, the 
method includes step 1 10 of receiving a set of media streams, wherein the set includes 
first type media stream components and second type media stream components. The first 
type media stream component can be modified by a modification process of step 120, 
while the second type media stream can not be modified. For example, the second type 
can be encrypted media stream components that are encrypted such as to prevent size 
and/or timing modifications The former usually cannot be statistical multiplexed rate 
shaped while the latter cannot be statistical multiplexed. 

[0086] Step 1 10 is followed by step 120 of applying a modification process that is not adapted to 
modify second type media stream components, such as to provide at least one modified 
first type media stream component. For example, the modification can include lossy 
compression and the second type media stream components can include encrypted media 
stream components. 
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[0087] Step 120 may include modifying the size and/or timing of the media stream components. 

[ 0 0 8 8 ] It is noted that the second type media stream components can be indirectly influenced by 
the modification of the first type media stream components. For example, by modifying 
the size of first type media stream components that are eventually multiplexed with the 
second type media stream components the timing of the second type can be altered. 
According to various embodiments of the invention the multiplex can be formed such as 
not to alter the timing of second type media stream components, such as by defining 
timing constraints that limit the allowed time shifting of second type media stream 
components. 

[0089] According to an embodiment of the invention step 120 involves executing modification 
sessions in a periodical manner. Each modification session may be associated with a 
group of media stream portions that are either 

(i) of a certain aggregate size, (ii) that are received during a certain time period, or (iii) are 
received during a certain time period. Usually, each modification session is responsive to 
a bit rate target that reflects available transmission bit rate during a certain time period. 
The bit rate target may be either provided or estimated. The modification session may 
also be responsive to media stream component modification priorities and even to an 
allocation of available bit rate per media stream component transmission priority. 
Conveniently, at least one modification session includes modifying the size/timing of at 
least one media stream component of the group, evaluating the size/timing of the at least 
one modified non-encrypted media stream component, and determining whether 
additional modification is required. The additional modification may involve modifying 
another non-encrypted media stream component of the group or re-modifying a modified 
media stream component of the group, or modifying the timing of encrypted components. 
[0090] The modification is preceded by differentiating between encrypted media stream 

components and non-encrypted media stream components. This differentiation may be 
responsive to the presence of an encryption indication associated with one or more media 
stream components. 

[0091] Step 120 is followed by step 130 of multiplexing the modified non-encrypted media 

stream components and the encrypted media stream components. If during step 120 some 
non-encrypted media stream components are not modified they are also multiplexed. 
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[0092] Step 130 may be followed by optional step 140 of determining at least one control 

parameter. This at least one control parameter may be applied to future media stream 
components - those who were not processed during the current modification session or 
even those who were not yet received. Step 140 may include determining an encrypted 
media stream component target bit rate. This determination can be responsive to various 
parameters such as: target bit rate, the bit rate of encrypted media stream components and 
of non-encrypted media stream components previously received during step 1 10. 

[0093] Step 140 is followed by one of the following optional steps: step 150 of selecting an 

encrypted representation of a media stream out of a set of encrypted representations that 
differ by their size; and step 160 of altering an encryption parameter of media streams. 
Conveniently, steps 150 and 160 are followed by either step 120 or step 110. 

[0094] According to an embodiment of the invention encryption priority may be assigned to 

media streams. Typically, if such priorities are defined then step 160 is responsive to the- 
encryption priorities. Usually, step 160 of altering encryption parameters has greater 
meaning when statistical multiplexing rate shaping is applied and when the different 
encrypted versions differ by size. 

[0095] Figure 4 illustrates a method 200 for partial encryption, according to an aspect of the 
invention. Method 200 receives a media stream (step 210) and converts it to multiple 
layers during step 220. The conversion may provide temporal scalability or spatial 
scalability, and may include applying multiple quantization levels and/or by applying 
filtering or manipulation. 

[0096] Step 220 is followed step 230 by encrypting at least a portion of one of the layers. 

[0097] Figure 5 illustrates a method 300 for partial encryption, according to an aspect of the 
invention. Method 300 starts by step 310 of receiving multiple layers that represent a 
media stream. The multiple layers can provide temporal scalability or spatial scalability, 
and be generated by applying multiple quantization levels and/or by applying filtering or 
manipulation. Step 310 is followed by step 320 of encrypting a portion of at least one 
layer. 

[0098] According to an aspect of the invention a controller (referred to 20 in Figure 1 ) may 

receive indications about the available bit rate as well as the bit rate of non-encrypted and 
encrypted media stream components and can balance between the modification manner of 
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the non-encrypted media stream segments, encryption of media streams and the 
representation of a media stream. The controller 20 may be responsive to modification 
priorities, encryption priorities and additional policy rules relating to the generation and 
transmission of media stream representations. For example, conditional access media 
streams that are more significant to users can be slightly modified, highly encrypted and 
also be represented by more than a base layer. 

[0099] The present invention can be practiced by employing conventional tools, methodology 
and components. Accordingly, the details of such tools, component and methodology are 
not set forth herein in detail. In the previous descriptions, numerous specific details (such 
as a certain compression standard) are set forth in order to provide a thorough 
understanding of the present invention. However, it should be recognized that the present 
invention might be practiced without resorting to the details specifically set forth. 

[00100] Only exemplary embodiments of the present invention and but a few examples of its 

versatility are shown and described in the present disclosure. It is to be understood that 
the present invention is capable of use in various other combinations and environments 
and is capable of changes or modifications within the scope of the inventive concept as 
expressed herein. 
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